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ABSTRACT 

In recent assessments, the National Center for 
Education Statistics (NCES) has permitted schools to exclude some 
students with disabilities or limited English proficiency because it 
believed that its assessments would not accurately measure the 
ability and achievement levels of these students. The exclusion of a 
portion of students from assessments by the NCES has raised doubts 
about the validity of its data. This report discusses the data 
validity concerns related to the current policy and explains how the 
NCES assesses and surveys students with disabilities and limited 
English proficiency. Approximately 77^ of students with disabilities 
attend special schools and thus are not assessed by the NCES. 
However, about 507. of students with disabilities are currently 
assessed. Alternative ways to collect outcome data on students with 
disabilities who are currently excluded include making accommodations 
to the NCES assessment procedure by, for example, providing 
additional time to complete the test or providing a separate room in 
which to take the test; conducting portfolio assessments; and 
collecting background data from the schools about students' 
achievement and abilities. The 1995 field test of the National 
Assessment of Educational Progress will examine new procedures for 
assessing students with disabilities or with limited English 
proficiency. (Contains 12 references.) (liLD) 
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Foreword 



Each year a large" number of written documents are generated by NCES staff and 
individuals commissioned by NCES which provide preliminary analyses of survey results and 
address technical, methodological, and evaluation issues. Even though they are not formally 
published, these documents reflect a tremendous amount of unique expertise, knowledge, and 
experience. 

The Working Paper Series was created in order to preserve the valuable information 
contained in these documents and to promote the sharing of valuable work experience and 
knowledge. However, these documents were prepared under different formats and did not 
undergo vigorous NCES publication review and editing prior to their inclusion in the series. 
Consequently, we encourage users of the series to consult the individual authors for citations. 

To receive information about submitting manuscripts or obtaining copies of the series, 
please contact Suellen Mauchamer at (202) 219-1828 or U.S. Department of Education, Office 
of Educational Research and Improvement, National Center for Education Statistics, 555 New 
Jersey Ave., N.W., Room 400, Washington, D.C. 20208-5652. 



Susan Ahmed Samuel S. Peng 

Acting Associate Commissiuner Statistical Service and 
Statistical Standards and Methodological Research 

Methodology Division 



iii 



ASSESSING STUDENTS WITH DISABILITIES AND 
LEVUTED ENGLISH PROHCIENCY 



Prepared by 



James Houser 



Policy and Review Branch 
Data Development Division 
National Center for Education Statistics 



January 6, 1995 



Table of Contents 



Section 

Forword 

Table of Contents 

Acknowledgements 

1. DATA VALIDITY AND CURRENT POUCY 

2. CURRENT STATUS 

3. DATA VALIDITY AND ALTERNATIVE ASSESSMENT 

4. NEXT STEPS 

BIBUOGRAPHY 



ACKNOWLEDGEMENTS 

This paper was prepared by James Houser in the Data Development Division of the National 
Center for Education Statistics (NCES) under the general direction of John Ralph, Chief of 
the Policy and Review Branch. Overall Direction was provided by Jeanne E. Griffith, 
Associate Commissioner for Data Development. 

A number of people throughout NCES provided information or advice regarding the 
development of this paper. They include Emerson J. Elliott, the Commissioner of Education 
Statistics and Ron Hall, his Chief of Staff. They also include Peggy Carr, Andy Kolstad, 
Gary W. Phillips, Laurence Ogle, and Sharif Shakrani from the Education Assessment 
Division in NCES. Individuals from the Elementary and Secondary Education Statistics 
Division in NCES who provided input include Jeffrey Owings; Paul D. Planchon; and Peggy 
Quinn. In the Data Development Division, Dawn Nelson, Eugene Owen, and Lois Peak 
provided information and advice. 

Other individuals within the Department of Education reviewed the paper. They include 
Judith Leonard from the Office of General Counsel; Lou Danielson from the Office of 
Special Education and Rehabilitative Services; and Gene Garcia, the Director of the Office of 
Bilingual Education and Minority Language Affairs. 

Outside of NCES, many individuals commented on the paper. The individuals include James 
D. Ysseldyke and Martha L. Thurlow, respectively the Director and Associate Director of 
the National Center on Educational Outcomes for Students with Disabilities. In addition, the 
paper was reviewed by Lizanne Stephano from the University of Illinois at Urban-Champaign 
and both Randy Bennett and Paul Williams from the Education Testing Service. 

Although many individuals contributed toward the development of this paper, the author 
accepts full responsibility for the accuracy of all information contained in the paper. The 
author, however, forewarns readers that some of the information in the report may become 
dated after August 1995. The 1995 field test for the National Assessment of Educational 
Progress will experiment with providing accommodations to assessments for students with 
disabilities and limited English proficiency. This experiment will be evaluated by August 
1995. 




ASSESSI NG STTTDKNTS WITH DISAB nTTTES AND 
f TMTTKD ENGL ISH PROFICIENCY 



This paper describes: 

1. what data validity concerns are related to the National Center for Education Statistics' 
(NCES) current policy of excluding some students with disabilities and limited 
English proficiency; 

2. how NCES currently assesses and surveys students with disabilities and limited 
English proficiency and what related research has taken place; 

3. what data validity concerns need to be considered in determining whether to conduct 
alternative assessments; and 

4. what steps NCES is taking to begin modifying how it assesses students with 
disabilities and limited English proficiency. 



1. DATA VALIDTTY AND CURRENT POLICY 

In recent assessments, such as the 1994 administration of the National Assessment of 
Education Progress (NAEP), NCES permitted schools to exclude some students with 
disabilities and limited English proficiency. NCES did so because it believed that its 
assessments, as designed, would not accurately measure the ability and achievement level of 
some students with disabilities or limited English proficiency. The exclusion of a portion of 
students from NCES' assessments raises issues about the validity of its data. 



Res ponsibili t y of a Statistical Aeenc v in Education 

o As the government's education statistical agency, NCES has an obligation to provide 
information that can be generalized to represent various populations. When the data 
are not representative, NCES has, first, to acknowledge that fact so data users will be 
informed, and second, to take steps to remedy the deficiency. 

Section 421 (c)(3) of the Perkins (vocational-technical education) Act requires the 
Secretary of Education to: 

ensure that appropriate methodologies are used in assessments of 
students with limited English proficiency and students with 
handicaps to ensure valid and reliable comparisons with the 
general student population and across program areas. 

We interpret this to apply to both vocational and non-vocational students. Still, the 
Administration's goals for inclusion set the government policy context that NCES 
must follow. 



• 



Implications of Inclasion and Exclusion for a Statistical Agency 

o Equity and inclusion traditionally involve equal access to benefits. Participation in 
NCES' surveys and assessments is not a direct benefit for an individual, rather it is 
often viewed as a burden. Since NCES' studies sample students and a school official 
excludes students (if they meet established criteria and are viewed by the school to be 
incapable of taking the assessment) prior to selecting the final sample, students are not 
stigmatized before their classmates by being excluded. Neither the excluded students 
nor their classmates know that any particular student is being purposefully excluded. 

o However, the inclusion of a group, such as students with disabilities or limited 

English proficiency, in NCES' studies could be construed as a benefit to that group if 
the study accuratelv measures their educational outcomes . Policymakers often 
determine resource levels and policies based on national data. 

o Since NCES is part of the U.S. Department of Education, the actions it takes to 
include or exclude students with disabilities and limited English proficiency provide 
examples for states, school districts, and schools to follow in their own assessments. 

o Inclusion raises issues of statistical bias and interpretation. 

► A study that excludes a subpopulation potentially biases the results of the 
study. Students with relatively severe disabilities or limited English 

• proficiency are probably more likely both to be excluded from NCES' 

assessments and to have lower assessment scores than students who are neither 
severely disabled nor severely limited English proficient. As a result, NCES 
acknowledges in its reports, but not always in a prominent manner, that these 
students are excluded to inform its data users that its data are not generalizable 
I to the excluded students. 

► A study that measures the outcomes of students using a different method from 
that used for the general population (e.g., an accommodation of extra time for 
a student with a learning disability or a Spanish-language translation for a 

, student with limited English proficiency) may not be measuring the same thing 

as the general assessment. Some of these accommodations have led to 
overestimates of the ability and achievement levels of students receiving the 
accommodation. 

o Although the guidelines (discussed under "Current Status") for excluding a student 
from an assessment may not be ideal, altering the guidelines may prove difficult to 
implement. NCES conducts assessments as part of several of its programs: NAEP; 
the National Adult Literacy Study; the international studies program; and the 
longitudinal studies program. A portion of students participating in NAEP take an 
assessment mat is designed to provide long-term national time series data. This 
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assessment consists of test items that have not been changed in any of the years that 
are included in the time series. In addition, data from NCES' different longitudinal 
studies are sometimes compared to one another, thus they provide time series data as 
well as longitudinal data.' If the exclusion criteria were altered, NCES might not be 
able to make valid comparisons between years for which different criteria were used. 
Since the population being tested would no longer be identical, changes in the time 
series data could be either the result of actual changes in performance of students or 
the result of adding more students with disabilities and limited English proficiency to 
the sample. If the exclusion criteria are altered for NAEP, it may be necessary to 
retain the existing exclusion criteria for the long-term time series sample or, at least, 
to estimate the size of its effect. 

o Data on racial and ethnic groups may be biased because some students with 

disabilities and limited English proficiency are excluded from NCES' surveys and 
assessments. Minority students are overrepresented among students with disabilities 
and limited English proficiency. To the extent that minority students are also 
overrepresented among students who are excluded, the data reported on racial and 
ethnic groups are biased. 

2. CURRENT STATUS 

This section describes the current status of both the Office of Educational Research and 
Improvement's and the Office of Special Education and Rehabilitative Services' survey and 
research activities related to students with disabilities and students with limited English 
proficiency. 

Status of Statistical Activities 

Survey Guidelines 

0 Because paper and pencil assessments may not accurately reflect the achievement level 
of some students with disabilities, school administrators exclude some students with 
disabilities from NCES' assessments. Beginning with the 1990 NAEP, schools were 
given guidelines stating they may exclude a student with a disability if: 

► "the student is mainstreamed less than 50 percent of the time in academic 
subjects and is judged incapable of participating meaningfully in the 



'For example, if data on high school seniors are compared using two different 
longitudinal studies, the National Longitudinal Study of 1972 (which has data on 1972 high 
school seniors) and High School and Beyond (which has data on 1982 high school seniors), 
the data would be considered to be time series data since it would study a population with a 
similar characteristic (being high school seniors) at two different points in time. 
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assessment, 



QE 

► the lEP^ or equi^-alent group had determined that the student is incapable of 
participating meaningfully in the assessment." (ETS and Westat, Inc., 1990, 
p. 11-11) 

o Beginning with the 1990 NAEP, NCES instructed schools to exclude students with 
limited English proficiency from its assessments only if all the following conditions 
apply: 

► "the student is a native speaker of a language other than English; AND 

► the student has been enrolled in an English-speaking school for less than two 
years (not including bilingual education programs)^; AND 

► school officials judge the student to be incapable of taking the assessment". 
(ETS and Westat, Inc., 1990, p. II-U) 

o Schools are instructed to include students with disabilities and limited English 

proficiency if school staff believe the students are capable of taking the assessment. 
Schools are also instructed that when there is doubt, students should be included. 
(ETS and Westat, 1990, p. II-U) 

o Administrat'ons of NAEP prior to 1990 and the National Education Longitudinal 

Study of 1988 (NELS:88, one of the data sets in NCES' longitudinal studies program) 
relied on the judgement of school administrators as to whether or not the student 
could take the test. They provided less rigorous criteria for determining which 
students with disabilities and limited English proficiency should take the assessment. 
(Johnson et al., September 1990, p. 96 and NCES, August 1990, p. 7) In the Third 
International Mathematics and Science Study, the procedures that NCES began using 
for NAEP in 1990 will be followed for the U.S. The rule for other countries will 
permit the exclusion of students based on criteria that include disability status and 
ability to speak the language of the country, but not the exclusion of more than ten 
percent of a country's student population. (Foy and Schleicher, April 1993, p. 6-3) 



^lEP stands for Individualized Education Program; all students receiving special 
education are required to have these programs. 

^This provision means that a student can be excluded from the assessment if he or she 
has taken the subject being tested in English for less than two years. 



Background Information 



o Students who are excluded from NCES' assessments are also excluded from 

background questionnaires that are linked to these assessments. One exception to this 
rule is for Spanish-speaking students with limited English proficiency. NCES 
provided Spanish-language background questionnaires for students participating in the 
first- and second- NELS follow ups. (NCES, November 1993, p. 101) 

o However, NCES obtains some data on students who are excluded from its surveys 
and assessments from school officials who complete an abbreviated questionnaire for 
these students. In addition, they are included in NCES' transcript studies when these 
studies are conducted in association with NCES' surveys and assessments. 

Exclusion Rates 

o NCES generally includes only students in its surveys who attend public or private 
schools.* As a result, individuals who are incarcerated, dropouts, and students 
enrolled in special schools for the disabled are generally excluded from NCES' 
surveys and assessments. In addition, some students with disabilities or limited 
English proficiency in regular schools are excluded from NCF^' surveys and 
assessments. 

► Incarcerated students, who account for less than one percent of 17-year-olds, 
are generally excluded from NCES' studies. 

► Dropouts account for approximately six percent of 17-year olds and are not 
generally included in NCES' cross-sectional school-based studies. 

► Approximately seven percent of students with disabilities (less than one percent 
of the general student population) attend special schools and, therefore, are not 
included in NCES' studies. Even if these special schools were included in the 
sampling universe for NCES' surveys and assessments, they still might be 
difficult to include in NCES' studies. NCES studies are generally grade-based 
and many special schools are not graded. One exception to this rule is the 
NAEP national trend sample. This sample is age-based and could potentially 
include special schools for students with disabilities. 



■"Most public and private schools are included in the universe of schools from which 
NCES draws its samples. Schools designed to meet the needs of specific populations, such 
as special schools for individuals with disabilities, are sometimes not included in the 
universe. (NCES, March 1990, p. 20) 
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► Among eighth-grade students in 1992, nine percent had an Individualized 
Education Program (lEP), approximately one-half of whom were excluded. 
As a result, five percent of all eighth-grade students in regular schools were 
excluded because of disabling conditions, 

► Approximately three percent of all eighth-grade students in schools in 1992 
had limited English proficiency, of which approximately two-thirds were 
excluded. As a result, two percent of all eighth-grade students were excluded 
because of language barriers. (Spencer, 1994, p. 16) 

► If 17-year-olds are excluded at the same rate as eighth grade students are 
because of the severity of their disability or limited English proficiency, then 
approximately 15 percent of all 17-year-olds are systematically excluded from 
NCES' assessments. 

Status of Research Activities 

o Only limited research has taken place on how to assess students with disabilities and 
limited English proficiency. To date, no experiments using different approaches to 
assessing students with disabilities have been conducted. 

o The National Academy of Education (NAE) has produced two papers commissioned 
by NCES, Issues in the Development of Spanish-Language Versions of the National 
Assessment of Educational Progress and A Study of Eligibility Exclusions and 
Sampling: 1992 Trial State Assessment. The first paper addresses the feasibility of 
conducting NAEP in Spanish. This paper concludes that 1) NCES should conduct a 
pilot study to examine the technical implications of a Spanish-language assessment and 
2) NCES should try to design NAEP so that it will capture a broader range of 
students than it currently does. The paper on eligibility exclusion^ addresses the 
exclusion of numerous types of students from the 1992 NAEP Trial State Assessment. 
This paper concludes that: 

► Private school students should be included in NAEP 

► NCES needs to examine if it is cost effective to shift resources away from 
having a large sample size (to reduce sampling error) and towards including 
students who are relatively expensive to assess, such as students with 
disabilities and limited English proficiency (to reduce sampling bias). 

► NCES should either include dropouts in NAEP or indicate in its publications 
what proportion of students are dropouts. 
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The National Center on Education Outcomes (NCEO), which is supported by the 
Office of Special Education and Rehabilitative Services, has produced a literature 
review on assessing students with disabilities that includes the results of a survey of 
state assessment practices regarding students with disabilities, Testing 
Accommodations for Students with Disabilities: A Review of the Literature. 

NCEO held a small working conference on hov/ to assess students with disabilities in 
March 1994. The conference included experts on assessment and research of students 
with disabilities. Staff from NCES and the Office of Special Education and 
Rehabilitative Services were included in the conference. An overview of the 
discussions from the conference are published in Making Decisions About the 
Inclusion of Students with Disabilities in Large Scale Assessments, The 
recommendations made in the publication include that 1) NCES should try to develop 
more objective criteria for schools to use when making decisions about inclusion and 
exclusion and 2) NCES should conduct various research activities to investigate the 
data validity '.ssues related to including students with disabilities and limited English 
proficiency in NCES' assessments. 

NCES convened a conference on assessing students with limited English proficiency 
in December 1994. The conference included experts in assessment and research of 
students with limited English proficiency. The group cautioned about assessing 
students with limited English proficiency in their native language since these students 
are frequently not literate in their native language. In addition, the group expressed 
concern that translations need to be conducted carefully since 1) the difficulty of 
vocabulary varies from one language to another and 2) some languages, such as 
Spanish, have numerous dialects. 

The Office of Educational Research and Improvement funds the Center for Cultural 
Literacy and Second Language Learning at the University of California at Santa Cruz. 
This Center plans to conduct research on using portfolio assessments to assess the 
literacy of students with limited English proficiency. 

NCES has budgeted to conduct a study designed to study potential bias on math and 
science items related to language ability as part of its activities in NAEP. The study 
will also address how the accuracy of these assessments might be improved. 
Although this study is not directly related to the main thrust of this paper - how we 
might collect outcome information on more students with disabilities and limited 
English proficiency - it may help us improve the quality ol assessment data on 
students with limited English proficiency, 

DATA VALroTTY AND ALTERNATIVR ASSESSMENTS 



This section describes the relationship between data validity and: 



o providing accommodations to NCES' assessments, 



o conducting portfolio assessments, 

o assigning test scores to students based on background characteristics and teacher 
observations, and 

0 translating NCES' assessments into languages other than English. 
Students with Disabilities 

o Many students with disabilities who are currently excluded from NCES' assessments 
could be included with no accommodations at all. In fact, Steven Ingels from 
National Opinion Research Center believes that NCES might be able to assess 85 
percent of all students with disabilities with no accommodations. NCES is currently 
assessing only about 50 percent. 

o There are at least three alternative ways to collect outcome data on students who are 
currently excluded from NCES' studies: 

► make accommodations to NCES' assessment instruments, 

► conduct portfolio assessments, and 

► collect background data on students from school officials about their ability 
and achievement levels and assign test scores based on these data. 

We do not know which method would yield the most accurate estimate of a student's 
achievement level. Nor do we know which would yield test scores most comparable 
to scores from the general assessment. 

o A fourth approach may need to be developed for the most severely handicapped 

students for whom independent living skills, not academic achievement, would be the 
primary concern of special education instructors. 

o Making accommodations to an assessment could stigmatize a student. His or her 
classmates couk be aware that the student was receiving special treatment as a result 
of a disabling condition if ti:e student was not already receiving the same 
accommodations as part of his or her instruction. 
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Data Validity Issues 



Accommodations to Assessments 

o Accommodations fall into two different categories: 1) those that are very unlikely to 
affect test scores, such as providing a separate room in which to take a test, and 2) 
those that are likely to affect test scores, such as providing additional time to 
complete the test. 

o To the extent a test is time limited, the provision of additional time decreases the 
validity of the test score. A learning disability is the most common disabling 
condition and the most common accommodation made to assessments for students 
with this condition is addition;U time to complete the test. 

o The National Center on Educational Outcomes for Students with Disabilities recently 
conducted a literature review on providing accommodations for students with 
disabilities. The literature indicated that accommodations have been made in many 
instances (e.g., for Scholastic Assessment Tests (SATs) and State-mandated tests). 
The literature review conducted by this research center did not indicate that 
accommodations were made to any assessments that were used primarily for statistical 
reporting. Research on accommodations made to the SAT and the Graduate Record 
Examination (GRE) indicated that most accommodations did not present serious 
problems with data validity. The most common accommodation, providing additional 
time, did present predictive validity problems. The test scores for students who 
received additional time biased the data and overpredicted their postsecondary grades 
(the bias equaled approximately one-third of a standard deviation). That is, students 
who received additional time to take the SAT and GRE did not perform as well 
academically as their test scores predicted they would. Although this study does not 
prove that providing additional time for some students to complete NAEP would 
undermine its validity, it does indicate that extended time to complete NAEP needs to 
be studied carefully. 

o The test scores for students receiving different types of accommodations, which will 
vary depending on the type and severity of disability, may also need to be reported 
separately since they may not be psychometrically equivalent to one another. 

Portfolio Assessments 

o Portfolio assessments are considered by many individuals to be potentially more 

informative about a student's achievement level than paper and pencil tests, regardless 
of whether a student has a disabling condition. Portfolio f.ssessments have not been 
commonly used to conduct large scale assessments for sta:istical purposes. In NAEP, 
however, it has been demonstrated that methodologies can be devised that permit 
uniform measurement on a wide variety of student writing. If the collected work of 



students with disabilities falls outside the range of work that can be uniformly 
measured, then it will require separate reporting (even if all students, regardless of 
whether they have a disabling condition or not, are participating in the portfolio 
assessment). 

Assigning Scores Based on Background Characteristics and Teacher Observations 

o This approach requires that an empirical process be developed to establish the 

relationship between background variables and student achievement. This could pose 
a problem in NAEP since it does not currently collect information on key background 
variables that have been linked to achievement such as parents' income. 

o If test scores are assigned to students based or "conditioned" on their background 
characteristics, the scores could potentially be biased. The test scores would be 
conditioned on the test scores of students with similar background characteristics. 
There might not be either a sufficient number of students with similar characteristics 
or a sufficient number of detailed items to properly match similar students. Either 
problem could potentially bias the test scores. 

All Three Strategies 

o The test scores for students participating under any of these three scenarios will 

probably need to be reported separately from the test scores for other students since 
the assessments would not be psychometrically equivalent (i.e., identical scores for 
students who did and did not receive accommodations would not reflect identical 
achievement or ability levels, instead they would reflect that the assessment with the 
accommodations was either easier or harder than the assessment without the 
accommodations). NCES does not report data for a given population if the number of 
individuals in the sample is below 30 (62 for NAEP). 

A cell size of 30 (62 for NAEP) is sufficient to report on a given population, but it 
might not be large enough to make statistical comparisons between estimates. The 
standard errors for two numbers need to be small enough that apparently large 
differences will be statistically significant. The standard error increases as the cell 
size decreases. Data for tliese small cells might not be interesting to researchers if 
conclusions cannot be drawn about large differences between estimates because the 
standard errors are too large for the differences to be statistically significant.^ 

o Any of these three scenarios could potentially render the time series data invalid. A 



^If the numbers being compared are not inherently comparable, as may be the case when 
the assessment is not administered in a similar fashion, having a cell size large enough to 
compare two averages is unimportant. 
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portion of students participating in NAEP take an assessment that is designed to 
provide long-term national time series data. This assessment consists of test items 
and administrative procedures that have not been changed in any of the years that are 
included in the trend. Another portion of students participating in NAEP take an 
assessment that provides short-term time series data. In addition, data from NCES' 
longitudinal data sets are sometimes compared to one another, thus they can provide 
time series data as well as longitudinal data.* If accommodations are made or 
portfolio assessments are conducted as part of a study that is designed to produce time 
series data, students might not be taking tests psychometrically equivalent to the 
administrations of assessments that did not provide alternative assessments. As a 
result, NCES might not be able to make valid comparisons between years in which 
alternative tests were and were not permitted. In addition, even if the assessments 
were psychometrically equivalent, they could potentially disrupt the trend data because 
students with disabilities might tend to score lower and more of them would be 
included. The result might be declining test scores when achievement may be static 
or even rising. 

Students with Limited English Proficiency 

o Some psychometricians believe that assessments in different languages are not 
psychometrically equivalent. Therefore, test scores may need to be reported 
separately for each Language (as they are for the SAT). 

o Students with limited English proficiency could be assessed in their native language if 
knowledge about a subject is the primary policy interest of the study. The primary 
policy interest behind many education assessments, however, is the ability of U.S. 
students to compete economically. In the U.S., there are economic advantages to the 
individual that stem from fluency in English. 

o Spanish is by far the most prominent language spoken by students with limited 

English proficiency. So few students with limited English proficiency speak any other 
language that NCES would be unlikely to obtain sufficient sample size within a 
randomly-drawn national sample (unless the sample size for NCES' surveys increases 
dramatically) to report separate test scores for languages other than Spanish. NCES 
does not report estimates when the number of observations is insufficient because with 
so few observations (less than 62 for NAEP and 30 for its other surveys), estimates 
are of questionable accuracy. Also, Spanish is spoken in many different dialects. 



*For example, if data on high school seniors are compared using two different 
longitudinal studies, the National longitudinal Study of 1972 (which has data on 1972 high 
school seniors) and High School and Beyond (which has data on 1982 high school seniors), 
the data would be considered to be time series data since it would study a population with a 
similar characteristic (being high school seniors) at two different points in time. 
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NCES might not be able to administer just one Spanish test if the Department of 
Education decided to translate assessments. Moreover, students are not always 
literate in the language spoken at home and some languages are only spoken and not 
written. As a result, paper and pencil tests may be difficult to administer to students 
with limited English proficiency, even if the tests, are translated into a student's native 
language. 

o Translated assessments could disrupt the trend data because students with limited 
English proficiency would tend to score lower and more of them would be included. 
The result could be declining tests scores when achievement may be static or even 
rising. 

4. NEXT STEPS 

NCES is undertaking studies designed to include students with disabilities and limited 
English proficiency in NAEP as part of the 1995 NAEP field test. NCES will provide 
accommodations to these students to determine the feasibility of using accommodations to 
include them. NCES is also testing different inclusion and exclusion criteria to determine 
which criterion maximizes the number of students included in NAEP. 

Accommodations 

Limited English proficient students in grades 4 and 8 will be administered two types of 
assessments: 

o a Spanish-English side-by-side bilingual assessment and 

o a Spanish-only assessment. 

This will allow NCES to determine which type of assessment students find most useful. 
These assessments will be administered to students categorized as both limited English 
proficient and Spanish-speaking. Some of the tested students will be those who would have 
normally been excluded from the NAEP assessment; some would have normally been tested 
in English. Test questions are being translated into standard Latin American Spanish and 
NCES is evaluating the accuracy of the translation. (ETS and Westat, Inc., 1995) 

As part of the 1995 field test, NCES will examine the feasibility of a variety of 
accommodations for students with disabilities including: Braille, large print, one-on-one 
administrations, extended testing time, multiple administrations, oral administration, and oral 
answering. All assessments will be conducted by the student's normal facilitator. For the 
Braille sessions, students will be given special instructions several days ahead of the 
assessment on the use of either a talking calculator or a Braille calculator and will be given 
one to use during the assessment session. (ETS and Westat, Inc., 1995) 
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Inclusion and Exclusion Criteria 



The 1995 NAEP field test will examine two different sets of procedures for students with 
disabilities and two different sets of procedures for students with limited English proficiency 
that are designed to maximize the participation rates of these students. 

Students with Disabilities 

One set of procedures will be used for students with disabilities participating in the math 
assessment and a different set of procedures will be used for those participating in the science 
assessment. 

o In the math assessment, students with disabilities will be included in the assessment 
unless their lEP prohibits their participation in assessments. Those students whose 
lEPs specify that they require an accommodation to participate in the math assessment 
will generally receive the accommodation.^ 

o Siucients with disabilities selected for the science assessment will not be assessed if 
their IE? prohibits their participation in assessments. In addition, students v/ho . 
require an accommodation to participate in the science assessment will not be 
included. (ETS and Westat, Inc., 1995) 

Students with Limited English Proficiency 

For students with limited English proficiency, one set of procedures will be used for the 
math assessment in grades 4 and 8 and a different set of procedures will be used for the math 
assessment in grade 12 and the science assessment in grades 4, 8, and 12. 

0 Students with limited English proficiency ^-lected to participate in the math 
assessment in grades 4 and 8 will be assessed in Englich if they have received 
instruction in English for three or more years. Students w^o have received 
instruction in English for fewer than three years will be included if the school 
determines that they can participate meaningfully in the assessnent in English or 
Spanish. These students will receive the assessment in the lang uage in which they are 
believed to be strongest. If a school decides that a student is str')ngest in Spanish, the 
student will receive either an assessment in Spanish only or an assessment that 
presents Spanish and English versions of the assessment in a side- by-side fashion. 



^Students will not always receive the accommodation in their lEP. For instance, there 
will not be a Braille version of the math assessment in grade 4 since many blind students are 
not able to read Braille at this grade level. 
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Students with limited English proficiency selected to participate in the math 
assessment at grade 12 or the science assessment at any of the three grade levels will 
be assessed in English if they have received instruction in English for the past three 
years. If these students with limited English proficiency have not received instruction 
in English for this time period, schools will be asked to determine if they can 
participate in the assessment in English. If schools determine that a student can 
participate, then he or she will be included, if not, the student will be excluded. (ETS 
and Westat, Inc., 1995) 
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